Compiling and Optimizing Dynamic Parallel Programs

Data parallelism is an array based programming model that achieves massive parallelism through the lock-step execution of individual instructions simultaneously on all members of a parallel array. The semantics of data parallel programming languages are designed for extremely fine grained parallelism (on the level of a single arithmetic instruction), with tight synchronization. Data parallel languages provide a high level model of parallel programming which is easy to understand, and which simplifies the process of writing and debugging parallel applications, in addition to providing scalability. This has allowed scientific programmers to write code that runs efficiently on a variety of high performancearchitectures,which has caused data parallelism to become the favored paradigm for scientific computation. Unfortunately, the underlying assumptions that synchronization and communication are inexpensive, and the limitation to simple array based parallel data structures makes data parallel programming languages poorly suited for writing non-scientific, general purpose parallel programs.